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ESP TESTS: THE PROBLEM OF STUOENT BACK6R0UNO OISCIPLINE 



J C Alderson (University of Uncaster) and A H Urquhart (Unlversl^ of 
Aston) 



This paper reports on three studies designed to Investigate the ef- 
fect of students' background discipline - I.e. knowledge of a particular 
academic area - on their comprehension of written texts. One of the 
principal motivations underlying the studies was the wish to collect 
empirical data on the necessity or desirability of ESP proficiency tests 
for tertiary level students coming to study In an English-speaking 
country. 

The studies were as follows: 

a) Study 1: the pilot study; at Aston University; five short texts 
gap-filling task. 

b) Study 2: at Aston and Lancaster Universities: same five texts; gap- 
filling and questions/short answers. 

c) Study 3: at Aston and Lancaster Universities; different versions of 
the British Council ELTS Studv Skills Nodule (MI). 

Study 1 ; An account of this study has already appeared In print 
(Alderson and Urquhart 1983). The description given here will there- 
fore concentrate on those parts of the study which also relate to 
Study 2. 

Subjects . The students taking part were studying on an English 
for biudy Skills Course at the University of Aston. They fell Into four 
groups depending on the academic area In which they were going to pursue 
their post*qraduate studies. The groups were 1) Development Admin- 
istration/Finance (15 students); 2) Engineering (11); 3) Maths and/ 
or Physics (6); 4) Liberal Arts (5). 

According to an Internal placement test, the first three groups 
were of very similar linguistic proficiency; the fourth group was more 
proficient. 

Texts: There were five texts In all, two Engineering, two Oevelop- 
ment Adrain/Fi nance, and one 'general' text. The Engineering texts 
(titles have been supplied for reference purposes) were 'Turbines', from 
an article in an Engineering periodical (Hulme, 1981) and 'Electrolytes', 
from an academic monograph (Gregory 1972). The Development/Finance 
texts were 'Malaysia' and 'Polanyl'* both from a university text*book 
(Latham 1978). The 'general' text was 'Ouixote', one of the 'Rate- 
builder' texts from the SRA 3B kit (Science Research Associates 1963). 
According to the publishers, this kit is designed for high or Junior- 
high school Li students. 

Each text was between 250 and 280 words long. 'Electrolytes' in- 
cluded a diagram with captiont and 'Malaysia' a table. The Fog Index 
for each text was between 17.5 and 18.5. 
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Tasks : A number of lexical Itens (between 15 and 22) were deleted 
from each text. In the judgement of the authors* all were restorable in 
terms of the text itself, wUhout, that 1s» appeal to knowledge outside 
the text. An effort was made to choose Items whose restoration de- 
pended on an overall grasp of the text» rather than on nearby cotertual 
Items. In the case of 'Electrolytes* and 'MeilayslA*, certain Items could 
be restored only after consulting the diagram or table respectively. 

Given that the object of the experiment was to test the effect of 
background knowledge. It might appear paradoxical to Insist that all 
Items should be restorable on the basis of the text. Anything else, 
however, would have been trivial. The ability to restore the missing 
Item In the text, **The battle of Hastings took place In the year ... 
clearly depends on background knowledge, but In an uninteresting way. 
It would seem that one of the assumptions underlying traditional 
'general' tests Is that the Information required to be extracted from 
the text is equally available to all readers. Since It Is one of the 
major hypotheses of this experiment that In spite of such avallablll^, 
certain groups will still be advantaged by a particular text. It Is 
clearly essential that In the experiment, all the required Information 
should at least In theory be equally accessible to all subjects. 

Results ; Detailed results can be found In Alderson and Urquhart 
(op.cit). They can be summarized here as follows: 

1) on the two Engineering texts, the Engineering group did better 
than the Development Admin/Finance group. 

2) similarly, the Development Admin/Finance group outperformed the 
Engineers on the two Development texts. 

3) there was a marked similarity between the performance of the 
Engineers and Physlcs/Haths students, on the one hand, and the Develop- 
ment and Liberal Arts groups on the other. The latter two groups out- 
performed the Engineering and Physlcs/Miths group on 'Quixote*, the 
'general ' text, suggesting that Engineering and Science students may 

be disadvantaged by such texts. 

4) there was a marked text effect. Overall, 'Turbines' proved to 
be the easiest text, and 'KTTanyl' and Quixote* the most difficult. 
There was no reason to predict this: the only measure of linguistic com- 
plexity used, the Fog Index, had Indicated that all five texts were all 
at the same level. At this stage, one should perhaps refer, not to 
text difficulty, but to texl x method . Indeed, without a satisfactory 
correlative of linguistic complexity, this is perhaps all that can ever 
be claimed. The text effect did not affect the hypothesis; it may, 
however, be relevant to the Interpretation of the rest *Vi of Study 2, as 
discussed below. 

Study 2 : This fell Into two parts, namely: 

1) an exact replication of Study 1 

2) a partial replication using the same texts but a different test 
task , namely Short Answers. 



4 



- 3 . 



The reason for replicating the Pilot Study Is obvious - the numbers 
Involved there were very small, too small, in fact, to justify calcu- 
lating significance values. The reason for extending the Pilot Study 
by adding another test task is also obvious: without such an ex- 
tension, the study is vulnerable to method effect. 

Subjects ; students on ESS courses at both Aston and Lancaster 
University took part. The groups were very similar in academic dis- • 
cipline to those participating in Study 1, namely: 

1) Development Admin/Finance (Aston) and Economics (Lancaster); from 
now on referred to as OAFE. 

2) Engineers (ENG) 

3) Science and Haths (SiM) 

4) Liberal Arts (L/A) 

Nuabers of subjects doing (a) gap-filling (b) Short Question/Answer, 
and (c) both are shown in table 1. 

Table 1 



Gap-filling Short Answer Both 





Aston 


Lancaster 


T 


Aston 


Lancaster 


T 


Both 




OAFE 


18 


8 


26 


24 


9 


33 


16/8 


24 


ENG. 


17 


4 


21 


16 


4 


20 


16/4 


20 


S M 


32 


6 


38 


28 


6 


34 


24/6 


30 


L/A 


13 


9 


22 


10 


16 


26 


10/9 


19 



The Gap-filling test was administered first; the Short Answer test 
about a week later. 

Results : The Gap-filling test was marked three times, once for 
exact restoration, once for exact restoration, once for synonyms, 
and once for any response judged acceptable in the general context. 
Results are presented in Table 2. 
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TABLE 2 

A. Gap-filling (Variable N) 



Economics Engineering 
Polanyl Malaysia Turbines Electrolytes 



Quixote 







a 


b 


c 


a 


b 


c 


a 


b 


c 


a 


b 


c 


a 


b 


c 


DAFE 


X 


9 


31 


43 


17 


33 


39 


33 


47 


56 


19 


34 


40 


10 


28 


33 




SD 


9 


23 


22 


6 


16 


22 


14 


20 


21 


12 


18 


16 


9 


18 


20 




X 


3 


14 


27 


10 


23 


24 


32 


49 


52 


21 


40 


47 


5 


18 


25 


SD 


4 


12 


18 


9 


17 


19 


17 


19 


19 


12 


16 


16 


7 


12 


14 


SaM 


X 


6 


17 


27 


20 


31 


33 


30 


42 


47 


28 


47 


54 


7 


20 


22 


SD 


6 


13 


17 


7 


11 


14 


15 


21 


20 


11 


13 


15 


6 


11 


T4 


L/A 


X 


13 


30 


54 


21 


33 


34 


43 


61 


63 


22 


37 


44 


16 


3a 


t 


SD 


12 


19 


24 


14 


23 


21 


13 


17 


23 


16 


21 


21 


1 







a- exact 
b- synonym 
€• acceptable 
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Gap-fiU significant differences between laea* - t-test) t-eiuct b -synonym c -acceptable 



1) Polanyi 




^N6 






SSM 






L/A 






a 




c 


a 


D 


r 


a 


b 


c 


OAFE 

ENG 

StN 


NS 


PC*06 


p<.c:^ 


NS 

m 


pi.05 
NS 


o<.0l 


NS 

PC.05 
PC.05 


NS 

Pi.05 
PC.05 


NS 

pc.O* 
PC. 001 


2) Nalaysia 




ENG 






SM 






L/A 






a 


b 


c 


a 


b 




/ 


b 


c 


OAFE 
ENG 

sm 


NS 


NS 


PC. 06 


NS 

PC. 01 


NS 
NS 


NS 
NS 


NS 


NT. 
NS 


NS 
NS 


3) Turbines 




ENG 






saH 






L/A 






a 


D 


c 


a 


D 


c 


a 


b 




OAFE 

ENG 

S<H 


NS 


NS 


NS 


NS 
NS 


NS 
NS 


NS 
NS 


NS 
NS 

Pc.05 


NS 
NS 

Pc.05 


NS 
NS 

Pc.05 


4) Electrolytes 




ENG 






SM 






L/A 






a 


b 


c 


a 


b 


c 


a 


b 


c 


OAFE 

ENG 

SIM 


NS 


NS 


NS 


NS 
NS 


Pc.05 
NS 


PC. 01 

NS 


NS 
NS 
NS 


NS 
NS 
NS 


NS 
NS 
NS 


5) Quixote 




ENG 






saH 






LyA 






a 


b 


c 


a 


b 


c 


a 


b 


c 


OAFE 
ENG 


NS 




NS 


NS 
NS 


NS 
NS 


Pc.05 
NS 


NS 

PC.Ol 
Pi. 01 


NS 

Pi. 01 
PC. 01 


NS 

Pi. 01 
PC. 001 
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The lollOMing points encrge: 

1) The DAFE group did significantly better on the Ot >.<opm»nt texts 
than did the Engineers, although on 'Malaysia', there nc difference 
between DATE and SiM. 

2) However, on the Engineering texts, the Engi . t not out- 
perfona the DAFE group, although SaM did significant- :^iUr on 
'Electrolytes'. 

3) L/A did significantly better than either SaH <^ tht" 'Quixote' 
Results for the Short Mswer Ust are present ^. 

Table 4 

8. Short Answers 



OAFE 


Polanyl 
X 47.1 


Malaysia 
56.1 


Turblft 
53,8 


> * 


, 1 


SD 25.1 


ZZ.3 


17.1 






EN6 


X 24.5 


42. J 


58.1 






SO 22.7 


23.7 


24.1 




;TT 


SaM 


X 34.9 


46.1 


58.0 


4,3 


40.8 


SO 22.3 


20.P 


16.4 


18.6 


21.3 


L/A 


X 55. 7 


54.4 


60.3 


46.6 


70.8 


SO 22.6 


20.0 


20.2 


19.4 


14.3 



gnlflcant differences between groups are shown In Table 5. (See over). 

The pattern of results Is similar to that shown by the scores on 
Gap-filling. On the two Development texts. OAFE performed significant- 
ly better than ENG. However, there was no significant difference 
between DAFE and ENG on the two Enqlneerlna texts. The L/A aroup out- 
performed DAFE and ENG on 'Polahyv and all other groups on ^Quixote'. 

If one compares results under the two test methods . the following 
points emerge: 

\) there was a strong "^^hod effect* AUhotah the overalt cor- 
relation between scores ;^ ,l]rtests Is quite hioh (.78). correlations 
between scores on partlcul « t<»«ts vary wicielv. Another sign of method 
effect Is that the rank ordti if text difficulty Is not the same under 
both methods. On Gap-filling. 'ElectfX)lytes' is a comparatively easy 
text, ranked second overall; 'Quixote' U difficult, ranked fourth. 
On Short Answers » however » these ranks an reversed. 
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Table S 

Slgnlft'^nt difference* between means (t-te:t) 
B. Shcrt answers 



1) 


Polanyl 


ENG 


SaM 




L/A 




OAFE 
SaN 


p .01 


NS 
NS 


P 

n 
r 


NS 
.001 

»UI 


2) 


Nil ays U 


ENG 


saM 




L/A 




OAFE 
SAN 


.06 


NS 
NS 




NS 
NS 
NS 


3) 


Turbines 


ENG 


saM 




L/A 




OAFE 

ENG 

SiN 


NS 


NS 
NS 




NS 
NS 
NS 


4) 


Electrolytes 


ENG 


SaN 




L/A 




OAFE 

ENG 

SaN 


NS 


NS 
NS 




NS 
NS 
NS 


5) 


Quixote 


ENG 


saN 




L/A 




OAFE 

ENG 

SAN 


NS 


p .05 
NS 


P 
P 
P 


.01 

.001 

.001 



2) In spite of this, the text effect found In Study 1 remains strong 
over both methcds, with 'Turbines' being the easiest text, and *Polanyr 
the most difficult. 

3) As In Study 1» the ENG and SIM groups performed very similarly 
(though not Identically). This was also true of OAFE and L/A. This 
finding, of course, may have an Important bearing on the ni««ber of ESP 
tests required. 

4) The main point that emerges under both methods Is that the 
hypothesis was only partially confirmed: the OAFE group outperformed the 
Engineers on Development texts « and the L/A group did better on 'Quixote* 
than any other group. However, the Engineers did not appear to be ad* 
vantaged by reading Engineering texts. 
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In part, thU My t>t due to differing degrtts of linguistic pro* 
ficiency. Prt-testing of students at both unlversltlts revealed tliat 
the L/A group tias the wst proficient. At Aston, the three other 
groups were at the saae level, but the Lancaster Ccononlsts were su* 
perlor both to £IIG and SaN. 

Linguistic proficiency nay go part*«ay to explaining the results. 
For the Lancaster students, correlations were available between scores 
on different texts, and scores on ELBA and CLTS tests. These cor- 
relations are set out In Table 6. 

Table 6 

Test correlations with ELBA and CLTS 



A. Short Answer N-35 N-20 





ELBA 


ELBA 


ELBA 


ELTS 


ELTS 


ELTS 




1 


2 


ToUl 


01 


G2 


Ml 


Malaysia 


.45 


.38 


.43 


.45 


NS 


.3B 


^lanyl 


.52 


.52 


.56 


.68 


.6g 


.69 


Turbines 


.42 


.50 


.50 


.55 


.41 


.49 


Electrolytes 


NS 


NS 


NS 


.44 


NS 


NS 


Don Quixote 


NS 


.45 


.40 


.61 


NS 


.49 


B. Gap-filling (AccepUble) 

ELBA N 21/25 
1 2 


ToUl 


ELTS 

N-11/13 
G1 


G2 


Ml 


Italaysia 


NS 


.34 


NS 


.55 


NS 


.54 


R>lafv1 


.53 


.61 


.63 


•67 


.51 


.71 


Turbines 


.57 


.66 


.66 


.73 


NS 


NS 


Electrolytes 


NS 


NS 


NS 


NS 


NS 


NS 


Don Quixote 


.50 


.46 


•51 


NS 


NS 


NS 



The highest correlations are between scores on the language tests and on 
*Po1anyi* and 'Turbines*. Under both Methods, the L/A group» linguis- 
tically the NDst competent, did better than the other 3 oroups. One 
night, then, envisage scores as being Mde up of (a) a linguistic pro- 
ficiency coinponentt (b) a backoround knowledge component. On Engineer- 
ing texts, the Engineers* proficiency component was smaller than the 
DAFE group, but their backoround knowledge component was greater» thus 
levelling scores. On Otvelopment texts» however, the D.Vf group was 
superior on both components, hence their higher scores. 
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Uhlit 11n9u1stk proflcltncy would deflnlttly %tm to hivt been « 
factor* iKMtvtri the txpUnitfon outllntd abovt will not sund. It Is 
tnit that In tht Wi group* tht Uncasttr Ccononlsts stM to h«v€ 
bctn nort proflcltnt; this m% not trut. hOMtvtr. of tht Aston 
Otvolopntnt Aa»1n/rin«nct oroup, who fonwd tht Itrgtr ptrt of tht Wt 
qroup ts t wholt. A ctlcuutlon btstd on tht Aston group tlont In- 
dlctttd thit thtrt was still no difftrtnct bttwttn thtn and tht 
Cnglnttrs on tht Cnglnttring Uxts. whllt thty did bttttr than tht 
Cnglnttrs on tht Oivtlopatnt ttxts. Straight linguistic proflcltncy 
Is not tnough. 

A rathtr mm coMpltx txplanatlon Involvts linguistic prof1c1tncy» 
background linoii1tdgt» and tht rank ordtr of Uxt difficulty. On tht 
Gap*f1111ng ttst. In both Study 1 and 2. *1iirb1nts' was tht tasltst ttn 
ovtrall. with 'Ptctrolytts* SKond. Thtst art» of courst. tht two 
Englnttring Uxts. Ont could argut that on linguistically tasy Uxts. 
oroups at tht stM Itvt) of proflcltncy would scort about tht saat. 
As tht linguistic difficulty of tht ttxts Incrtastd. scorts bastd on 
proflcltncy would ttnd to drop away, and bt rtplactd by scorts achltvtd 
through bacground knowltdgt* If that was avallablt. Thus on an 'tasy' 
ttxt. Iikt ^Turb1nts\ groups at tht saat Itvtl of linguistic pro- 
flcltncy. t.g. CN6» DAFE and SaM, would scort roughly tht saa« (tht L/A 
group would scort hightr). On linguistically «ort difficult ttxts. 
t.g. 'Polai^ri im and SaN would bt Itft bthlnd. This dots stt« to tx- 
pt' n soat rtsults. On 'Turblnts*. txact scoring, tht scorts wert 
3C SaN). 32 (CNG). 33 (OAFC). and 43 /L/A). 

Kf^wtvtr. ntat though such an explanation My bt. It cannot covtr all 
tht data* Oi Gip-f1111ng. *C1tctro1ytts' was an tasy ttxts on Short 
Answers, howevtr. It was conparatilvtly difficult. Mortovtr. If 
It was IV^ulstlc Sl«p11c1^ that Mdt 'ClKtrolytts' tasy on Gap- 
filling, on^ ^^^uTd txptct a high corrtlatlon bttwttn scores on that 
text and BA and CLTS. Exanlnatlon of Table 6 shows that this was 
not the ca<^ .. On both Short Answer and Gtp-f1111ng» correlations for 
this parti ular tey ^ lower than for any other. 

Cone Its Ion : The evidence Is that background knowledge has a sig* 
niflcant effect. Ih the evidence of Study 2 scores, however. It does not 
seen to operate synetrlcallys Engineers were disadvantaged by being 
tested outside their speciality • the DATE group were not. No satis- 
factory explanation has been found for this asyiaetry. 

Study 3 : The test-tasks used In Study 1 and 2 had certain advan- 
tages. Botn Gip-f1111ng and Short Answer tests are easy and quick to 
construct and adtolnlster. and reasonably easy to score. However, they 
are both open to charges of subjectivity. Pre-tests with suitable 
native-speakers. Including subject specialists, and Uen- analyses of 
their responses, would help to Mke the tfsts less subjective. How- 
ever, such procedures would. In effect^ conpllcate tests whose major 
justification Is their .slmpllcltv. Moreover, they would not help to 
Improve the construct or face validity of such tasks used as pro- 
ficiency tests. 

We therefore decided to conduct a studv uslno groups of students 
similar to those who took part In Studies 1 and 2. but replacing t*** 
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orl9lni1 ttst Utkt m\th « aodult (Kl) of tlit IrUlth Councn tin 
ttst (1). U1I119 part of tht CITS Utt Md tfit fo110w«A9 AdvMUiKS* 

1) tlw CITS ttst U « sophlstlcatod ttst, bastd on « co«p1t«t 
soMnHit fioii«o«plrlc«1 fittds «fii1|Sls (IMy)i 197S). 

2) U Is tht proflcltncy ttst ustd now by Uit Irltlsli Council for 
ustssiM tht UngMiOt proflcltncy of pottfitUI scholtrshlo stodtnts, 
and htfict ut by a larot proportion of ttrtlary-ltntl fortli^ <ttiidtnts 
iflsh1n9 to stiidly In IrlUln. 

3) tht studyskllls aodult (HI) conUlns rttdlno and stu4y*skl11s 
Mttrtal approprlaU for tht froups UMng part In Stydlts 1 and Z. 

mthod i Tht ttsts ustd utrt (a) 3 siAJtct-sptclfIc alttmatim 
of tKmrs N1 StMdy Skills f^lt, namly •Soc^fl ^ 
*Ttchnolo«y* and *Gtiitra1 ^^^t^icM (b)^tht 61 (Andlng) and 6? 
ILIsttnlng) Mdults of t^ i^tfS ttst. Tht ordtr of prtstnutlon ms as 
follows: 

1) Gl, G2 and N1 (Social Scltnct): at tht sUrt of tht CSS coorstt 
MStd for plactntnt purposts. 

Z) HI (Ttchnology): In Mttk « of tht S Mttk courst« uttk 2 of tht 
4 wttk. 

3) Nl (Gtntral Acadtsic): 2 Mtks laUri ustd as txit ttst. 



Siibjtcts ; studtnts btlongtd to tht s«m gtntral artas as btfort. 
PartTyMcaiit of tht conttrtinU Inpostd by tht txptrlatnUl dtsign, 
tnd tht nay tht dtslon ms adapttd to tht adilnlstratlvt rtqulrtmU 
of tht CSS coursts, tht Cnglnttrs and tht Scltnct/Miths oroups Mrt 
OMiblntd on this occasion. This ms Justlfltd by tht tvldtnct frw 
Studlts 1 and Z that thty art clostly rtlattd groups. Only subJtcU 
who sat all thrtt Ml ttsts wtrt counttd. Of thtst, 41 mrt OAFCt n 
SsM and C, and 41 L/A. 

Rtsults: Provisional rtsults. bastd on band scorts, art prtstnttd 
In TkhU 7. SInct Korts art still bting proctsstd, no signlflcanct 
valuts art Includtd. 
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study 3 
Table 7 



ELTS Scores 





G1 


G2 


HI (Soc Sc ) 


Ml (Tech) 


HI (Gen Ac) 


N 




X S.02 


S.74 


4.62 


S.12 


S.82 


41 


OAFE 












SD 1.(M9 


1.21S 


1.2S4 


1.6S4 


1.105 






X 4.8 


s.b 


4.49 


S.96 


S.73 


35 


S/HaE 












SD 1.476 


1.388 


1.300 


1.77 


1.314 






X S.68 


6.3S 


S.22 


S.4S 


6.37 


41 


L/A 












SD 0.914 


1.2 


1.280 


1.219 


0.873 






K 4.64 


S.11 


4.36 


6.21 


S.64 


14 


ENG 












SD 1.184 


0.944 


1.13 


1.282 


1.13 






X 4.9 


S.7b 


4.S7 


5.79 


5.79 


21 














SD 1.663 


1.S86 


1.417 


2.(M7 


1.445 





Discussion 

If the hypothesis had been conflnoed in all Its details, each group 
would have scored best on the subject alternative most appropriate to 
Its meinbers. I.e. DATE would have scored highest on 'Social Science*. 
S/H and ENG on 'Technology* and L/A on 'General Academic'. Actual 
performance on 'Appropriate', 'Inappropriate' and 'Neutral' tests is 
shown in Table 8 (for L/A, there Is no 'Neutral' test, although one 
might predict that they would do better on "Social Science' than on 
'Technology'). 

Table 8 



Group performance on Appropriate/Inapproprlate/Neutral Texts 





Approp 


Inapprop 


Neutral 


DAFE 


3 


2 


1 


S/H&E 


1 


3 


2 


L/A 

1 


1 


2/3 





13 



• 12 - 



With one exception I the table conflms the hypothesis and the as- 
sumptions underlying this part of the £ITS test, e.g. S/MaE did best 
on * Technology* I the mo;*t appropriate t«st, second best on ^General 
Academic', the 'neutral' text, and worst on 'Social Science', the 
Inappropriate* test. The exception Is the DATE group, who did worst on 
the 'appropriate' test. I.e., 'Social Science'. Apart from this, even 
this group conformed to the hypothesis, doing better on the 'Neutral' 
test than on the 'Inappropriate' one. 

Two explanations are Immediately available to account for this 
deviation from the hypothesis. The first Is that the 'Social Science' 
module Is simply more difficult than the others. Table 9 shows that all 
3 groupr found It hardest. 

Table 9 



• easiest) 


Soc Sc 


Tech 


Ger. Acad 


DATE 


3 


2 


1 


S/HaE 


3 


1 


? 


L/A 


3 


2 


1 



While, however, this may be true, there is another explanation. 
The results may be confounded by the presence of a learning effect. 
With the partial exception of the S<M and E group, the subjects 
found the first test the most difficult, the second easier, and the 
third easiest of all. One may, of course, entertain some doubts 
about such a learning effect. Mai\y of the students had attempted the 
Nl module before arriving on the courses they might have been ex- 
pected to be familiar with the general form and requirements. Hori>- 
over, whatever one's faith In the efficacy of either Aston or 
Lancaster ESS courses, one may doubt such a learning effect on such a 
complex test over a period, for the 4-week students, of 2 weeks. The 
possibility remains I however, and precludes firm conclusions being 
drawn about the relative difficulty of the 'Social Science' alternative. 
We are left with the fact that the S&M and E group performed better 
than the other two groups on the 'Technology* module, and this 
cannot be explalneaby a learning effect, ihey did this, moreover, 
in spite of the fact that scores on 01, G2, and the other Ml al» 
ternatlves sugqest that the other two groups were In general more 
proficient. If one separates the Engineers from the S/H and E 
group (as on Table 7), their superiority on the 'Technology' 
alternative Is even more striking. 

Conclusions 

The three studies reported above result In somewhat conflicting 
results. Study 1, without significance values. Indicated that the 
overall hypothesis was correct In virtually all respectss Study 2 
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suggested that Engineers were disadvantaged on texts outside their 
area, whereas DAFE subjects were not; Study 3 showed the opposite. 

Taken together, the studies do seem to confirm the Importance of 
background discipline. Njedless to say, further research Is needed 
to resolve the contradictions. 



Notes 



The authors would like to thank the British Council for making these 
tests available for the experiment. 
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